Economic and financial modeling and planning is commonly used to estimate or predict the performance and outcome of real systems, given specific sets of input data of interest. An economic-based system will have many variables and influences which determine its behavior. A model is a mathematical expression or representation which predicts the outcome or behavior of the system under a variety of conditions. In one sense, it is relatively easy, in the past tense, to review historical data, understand its past performance, and state with relative certainty that the system's past behavior was indeed driven by the historical data. A much more difficult task, but one that is extremely valuable, is to generate a mathematical model of the system which predicts how the system will behave, or would have behaved, with different sets of data and assumptions. While forecasting and backcasting using different sets of input data is inherently imprecise, i.e., no model can achieve 100% certainty, the field of probability and statistics has provided many tools which allow such predictions to be made with reasonable certainty and acceptable levels of confidence.
In its basic form, the economic model can be viewed as a predicted or anticipated outcome of a mathematical expression, as driven by a given set of input data and assumptions. The input data is processed through the mathematical expression representing either the expected or current behavior of the real system. The mathematical expression is formulated or derived from principles of probability and statistics, often by analyzing historical data and corresponding known outcomes, to achieve a best fit of the expected behavior of the system to other sets of data, both in terms of forecasting and backcasting. In other words, the model should be able to predict the outcome or response of the system to a specific set of data being considered or proposed, within a level of confidence, or an acceptable level of uncertainty. As a simple test of the quality of the model, if historical data is processed through the model and the outcome of the model, using the historical data, is closely aligned with the known historical outcome, then the model is considered to have a high confidence level over the interval. The model should then do a good job of predicting outcomes of the system to different sets of input data.
Economic modeling has many uses and applications. One emerging area in which modeling has exceptional promise is in the retail sales environment. Grocery stores, general merchandise stores, specialty shops, and other retail outlets face stiff competition for limited customers and business. Most if not all retail stores make every effort to maximize sales, volume, revenue, and profit. Economic modeling can be a very effective tool in helping the store owners and managers achieve these goals.
Retail stores engage in many different strategies to increase sales, volume, revenue, and profit. One common approach is to offer promotions on select merchandise. The store may offer one or more of its products at temporary sale price, discounts for multiple item purchases, or reduced service charges. One or more items may be offered with a percentage off regular price, fixed reduced price, no interest financing, no sales tax, or the well-known “buy two get one free” sale. The store may run advertisements, distribute flyers, and place promotional items on highly visible displays and end-caps (end displays located on each isle). In general, promotional items are classified by product, time of promotion, store, price reduction, and type of promotion or offer.
The process by which retailers select and implement promotional programs varies by season, region, company philosophy, and prior experience. Some retailers follow the seasonal trends and place on promotion those items which are popular or in demand during the season. Summertime is for outdoor activities; Thanksgiving and Christmas are for festive meals, home decorations, and gift giving; back-to-school is new clothes and classroom supplies. Some retailers use flyers and advertisements in newspapers, television, radio, and other mass communication media for select merchandise on promotion, without necessarily putting every item at a reduced price. Some retailers try to call attention to certain products with highly visible displays. Other retailers follow the competition and try to out-do the other. Still other retailers utilize loss-leaders and sell common items at cost or below cost in an effort to get customers into the store to hopefully buy other merchandise. The retailers may also focus on which other items will sell with the promotional items.
Promotional programs are costly and time consuming. Flyers and advertisements are expensive to run, base margins are lost on price reductions, precious floor-space and shelf-space are dedicated to specific items, and significant time and energy are spent setting up and administering the various promotions implemented by the retailer. It is important for the retailer to get good results, i.e. net profit gains, from the promotional investments. Yet, most if not all retailers make promotional decisions based on canned programs, gross historical perception, intuition, decision by committee, and other non-scientific indicators. Many promotional plans are fundamentally based on the notion that if we did it in the past it must be good enough to do again. In most situations, retailers simply do not understand, or have no objective scientific data to justify, what promotional tools are truly providing the best results on a time dependent per product basis.
Customers make their own buying decisions and do not necessarily follow trends. Retailers may have false understanding as to what factors have been primarily driving previous buying decisions and promotional successes. What has been perceived as working in the past may not achieve the same results today, possibly because the basis for belief in the effectiveness of prior promotion programs is flawed. Other unknown or non-obvious factors may be in play driving customer buying decisions which undermine or reveal the weakness in previous promotions. Economic, demographic, social, political, or other unforeseen factors may have changed and thereby altered the basis for customer buying decisions. In spite of costly and elaborate promotions, retailers not infrequently end up with disappointing sales, lower than expected profits, unsold inventory, and lost opportunities based on promotional guesswork. When a promotional program fails to achieve intended objectives, retailers, distributors, manufacturers, and promotional support organizations all lose confidence and business opportunity.
The retail industry as a whole operates on a low margin, high volume business model. Many merchandizing elements affect customer purchase decisions and consequently impact the volume of sales that any given retail outlet experiences. Price and promotion are probably the two key drivers of sales volume, with promotion playing an increasingly important role as retail margins and prices measured in inflation adjusted dollars drop. The margin pressure experienced by the retail industry is driven by many factors, most importantly the increase in large-box (e.g., Costco) and “every day low price” (e.g., WalMart) players in the industry.
Since margins are low, promotions almost always represent a drop in profit generated by the promoted items, in spite of the increase in sales caused by the promotional activity. Clearly, then, the motivation of retailers to participate in promotional activity is that the promotion items become “loss-leaders”, i.e., that the promoted items will either drive increased traffic to the store, and/or that the promoted items will drag other items along with the promotion purchase. This effect is termed “affinity” in the retail industry. The inverse effect, in which sales of non-promotional items are lost due to the purchase of promoted items in their stead is termed “cannibalization”. Characterization and prediction of the affinity and cannibalization (AC) effect are the central concerns of the method presented here.
The AC effect is widely believed in the retail industry to be a significant driver of dollars to the enterprise bottom line. Consequently, there is significant economic value in a model which can both characterize AC relationships and accurately predict fiscal impact of AC on promotional activity.
Past work in this area has centered on characterization of the AC effect between individual pairs of products. Consider two specific products Pi and Pj in store S1. Most schemes report variations on four basic metrics, upon which an analyst can make an inference as to the strength of the affinity relationship. These metrics are all simple aggregate probabilities based on the co-occurrence of two products in market baskets.
                    Confidence        =                  prob          ⁡                      (                                          P                i                            |                              P                j                                      )                                                            Reverse          ⁢                                          ⁢          Confidence                =                  prob          ⁡                      (                                          P                j                            |                              P                i                                      )                                                            Affinity          ⁢                                          ⁢          Metric                =                              prob            ⁡                          (                                                P                  i                                ⋀                                  P                  j                                            )                                                          prob              ⁡                              (                                  P                  i                                )                                      ⁢                                                  ⁢                          prob              ⁡                              (                                  P                  j                                )                                                                            Support        =                  prob          ⁡                      (                                          P                i                            ⋀                              P                j                                      )                              
These values are sufficient to infer if the actual co-occurrence is significantly greater than the expectation of co-occurrence for uncorrelated products, and also if the relationship is asymmetric. Asymmetric relationships are typical if one of the two products is the dominant seller, e.g., electric drills drive sales of drill bits but not the other way around, even though drill bits may be higher unit sellers than electric drills.
An alternative approach to characterization of relationships is obtained by examining temporal correlation of product sales, looking for correlated increases and decreases in sales between the hypothesized AC items. This approach may be statistically more sound, and might yield better characterizations than the previous method. However, it is also more computationally intense and therefore less practical given the large quantities of data that need to be processed to find relationships in retail sales data.
In order to plan effectively, e.g., choosing the optimal set of products and promotional attributes, it is necessary to quantify the units, sales, and profit projections and baselines for the planned promotion. Thus, any approach that serves only to characterize but not predict the effect of AC fails to fit the full requirements of a promotion planning system. Moreover the former method fails to capture correlations of cannibalization because it ignores the correlation between the driven product and the probability of not finding the driver product in the basket. The latter method fails because it does not examine actual co-occurrence data, and because causes of fluctuation which are co-linear cannot be well-resolved by simple time series analysis.
A need exists for an economic model which helps retailers make effective and successful promotional decisions in view of customer responses.